Simulation Algorithms for Regenerative Processes

This chapter is concerned with reviewing the basic ideas and concepts underlying the use of regenerative structure in the development of efficient simulation algorithms. While it has long been known that discrete state space Markov chains exhibit regenerative structure, we argue that well-behaved discrete-event simulations typically also contain an embedded sequence of regeneration times. However, algorithmic identification of the corresponding regeneration times turns out to be a nontrivial problem. We discuss the theoretical and implementation issues involved in identifying the corresponding regeneration times, and describe how regenerative methodology supplies effective solutions to several difficult simulation problems. In particular, we discuss the use of regeneration in the context of steady-state simulation as a means of efficiently computing confidence intervals and correcting for initial bias. We also point out that regeneration has the potential to offer significant algorithmic efficiency improvements to the simulationist, and illustrate this idea via discussion of steady-state gradient estimation and computation of infinite horizon expected discounted reward.